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FOREWORD 

The  National  Voluntary  Laboratory  Accreditation  Program  (NVLAP) , established 
in  1976,  is  administered  by  the  National  Institute  of  Standards  and  Technology 
(NIST)  formerly  the  National  Bureau  of  Standards  (NBS) . NVLAP  is  a voluntary 
system  for  assessing  and  evaluating  testing  laboratories  and  accrediting  those 
found  competent  to  perform  specific  test  methods  or  types  of  test  methods. 
Laboratory  Accreditation  programs  are  established  for  specified  product  or 
service  areas  in  response  to  requests  and  demonstrated  need. 

This  publication,  intended  for  the  NVLAP  technical  experts  (TEs)  who  serve  as 
assessors  and  evaluators,  describes  general  policies  and  practices  of  NVLAP 
assessment  and  evaluation.  The  specific  technical  criteria  for  assessing  and 
evaluating  laboratories  are  provided  elsewhere,  in  NVLAP  Handbooks  and 
checklists  for  each  technical  area. 
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NVLAP  ASSESSMENT  AND  EVALUATION  MANUAL 


I .  INTRODUCTION 

This  manual  describes  the  role  of  a NVLAP  Technical  Expert  (TE)  in  performing 
the  laboratory  assessment  and  evaluation  functions  of  the  National  Voluntary 
Laboratory  Accreditation  Program  (NVLAP) . The  information  covers  the 
responsibilities  of  a NVLAP  TE  as  a representative  of  NVLAP  who  must 
understand  all  aspects  of  the  accreditation  process,  including  NVLAP  policies 
and  procedures,  the  prescribed  sequence  of  events,  and  the  conduct  of  an  on- 
site assessment. 

The  NVLAP  accreditation  process  is  described  in  handbooks  specifically 
prepared  for  each  field  of  testing  in  which  accreditation  is  offered. 
Handbooks  are  included  in  the  application  packages  provided  to  prospective 
participants  in  a given  laboratory  accreditation  program.  Each  handbook  also 
identifies  the  specific  criteria  and  other  technical  requirements  for  that 
particular  program. 


II.  NVLAP  APPROACH  TO  ACCREDITATION 

NVLAP  provides  an  unbiased  third-party  evaluation  of  an  applicant 
organization.  Potential  clients  of  that  organization  can  consider  NVLAP 
accreditation  as  an  indicator  of  confidence  in  selecting  required  services. 

NVLAP  helps  both  laboratories  and  their  clients.  It  provides  assistance  and 
support  to  applicant  organizations  and  encourages  self-improvement.  Users  are 
supplied  with  information  about  accredited  organizations  so  that  they  can  make 
appropriate  choices  with  respect  to  the  services  they  need. 

The  NVLAP  approach  is  non-adversarial ; it  is  non-antagonistic , and  helpful, 
but  not  patronizing.  There  are  no  unreasonable  demands  nor  unnecessary 
requirements.  Although  conformance  with  the  accreditation  criteria  is  firmly 
required,  flexibility  is  exercised  by  assisting  applicants  in  achieving  that 
conformance . 


III.  ROLE  OF  NVLAP  TECHNICAL  EXPERTS 

NVLAP  TEs  carry  out  several  functions  in  the  accreditation  process.  They 
conduct  on-site  assessments  of  applicant  laboratories,  evaluate  pertinent 
information  (including  assessment  reports,  responses  to  reports  of 
deficiencies,  and  proficiency  test  results)  for  the  purpose  of  recommending 
appropriate  accreditation  actions,  and  they  are  a technical  resource  to  NVLAP. 

The  TE's  main  contribution  is  technical  expertise.  TEs  provide  technical 
advice  and  analyses  required  to  evaluate  a laboratory's  compliance  with  the 
accreditation  requirements.  NBS  technical  staff  and  others  may  also 
contribute  technical  expertise  at  various  stages  of  the  development  and 
implementation  of  NVLAP  activities. 
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Each  laboratory  must  be  evaluated,  as  uniformly  as  possible  in  relation  to 
others.  Assessments  and  evaluations  must  be  fair  and  even-handed. 
Interpretations  of  requirements  must  be  consistent  among  all  TEs  within  the 
same  technical  area.  If  doubts  arise  about  a requirement,  they  should  be 
discussed  with  other  TEs  as  well  as  the  cognizant  NVLAP  staff. 

On-site  Assessor 

As  an  on-site  assessor,  a TE  travels  to  the  laboratory  site  and  performs  an 
in-person  assessment  of  personnel,  equipment,  procedures  and  records.  In  this 
role,  a TE  represents  NVLAP  directly  and  is  expected  to  act  in  a friendly, 
courteous,  professional  manner.  Two  types  of  assessments  are  performed,  a 
regular  on-site,  where  all  aspects  of  the  laboratory  are  reviewed,  and 
monitoring  visits,  where  the  review  is  limited. 

Technical  Evaluation 

In  conducting  a Technical  Evaluation,  the  TE  reviews  the  complete  file  of 
information  gathered  on  a laboratory  to  determine  if  the  technical 
requirements  for  accreditation  have  been  met.  This  may  be  done  individually, 
in  a group  meeting  with  other  TEs,  via  conference  telephone  call,  by  mail  or 
other  appropriate  means.  The  result  will  be  a recommendation  to  accredit,  a 
request  for  further  information,  or  a recommendation  not  to  accredit. 

Technical  Resource 

Issues  requiring  technical  assistance  occasionally  arise.  The  NVLAP  staff  may 
not  include  the  requisite  technical  experts  and  hence  must  have  access  to 
appropriate  technical  resources.  These  resources  may  be  immediately  available 
from  an  NIST  technical  division;  otherwise,  the  NVLAP  TEs  for  that  technical 
area  will  be  called  upon  for  assistance. 

Conflicts  of  Interest 

Since  the  TEs  review  extensive  information  about  applicants,  perhaps  including 
sensitive  data  about  the  operation  competition,  or  other  business  practices  of 
the  applicant  organization,  it  is  vital  to  avoid  any  real  or  apparent  conflict 
of  interest.  If  a TE  knows  or  anticipates  that  there  is  or  may  be  a conflict 
of  interest  with  a given  assignment  it  is  incumbent  on  that  TE  to  advise  the 
appropriate  NVLAP  staff  immediately. 

IV.  PREPARATION  FOR  AN  ON-SITE  VISIT 
NVLAP  assignment  of  a TE 

The  assignment  of  TEs  to  perform  assessments  is  based  on  matching  individual 
expertise  with  an  applicant's  requested  scope  of  accreditation,  at  the  same 
time  avoiding  any  conflict  of  interest.  A TE  should  notify  NVLAP  immediately 
if  technically  unprepared  to  assess  or  evaluate  a laboratory  or  if  aware  of 
possible  conflict  of  interest.  Normally,  different  assessors  are  scheduled  on 
successive  visits  to  a laboratory. 
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For  the  first  one  or  two  assessments,  an  assessor  is  usually  accompanied  by  a 
NVLAP  staff  person.  The  assessor  subsequently  works  alone  although  a NVLAP 
staff  person,  or  possibly  a new  TE  in  training,  may  occasionally  accompany  an 
assessor . 

Laboratories  have  the  right,  with  appropriate  cause,  to  refuse  acceptance  of 
an  assessor.  Since  applicants  are  usually  informed  in  advance  about  the 
proposed  assessor,  any  objections  will  normally  be  considered  prior  to  the 
assessor's  contacting  the  applicant  or  going  to  the  premises.  In  the  event 
that  an  applicant  refuses  an  assessor  when  called,  or  at  the  door,  the 
assessor  should  notify  NVLAP  staff  immediately. 

When  a TE  is  assigned  to  perform  an  assessment,  NVLAP  will  send  copies  of  the 
applicant's  application  form,  any  proficiency  test  data,  the  last  assessment 
report,  any  deficiency  response  letters  from  the  applicant,  and  any  other 
pertinent  information.  These  documents  should  be  kept  in  confidence.  After 
the  assessment  has  been  completed,  the  TE  must  return  all  documents  to  NVLAP. 

An  assessor  typically  has  at  least  60  days  to  schedule  and  complete  the 
assigned  assessment(s) . A faster  response  may  occasionally  be  requested. 

Preparation 

An  assessor  must  prepare  for  an  assessment  by  thoroughly  reviewing  all 
information  and  organizing  the  time  available  during  the  assessment  to  the 
best  possible  advantage.  Inadequate  preparation  will  slow  down  the 
assessment,  create  an  unfavorable  impression  of  NVLAP  and  the  assessor,  waste 
the  applicant's  time,  and  render  the  results  less  valuable. 

Preparation  for  an  initial  assessment  (first-time  applicant)  may  be 
substantially  different  from  that  of  a renewal  applicant.  The  initial 
assessment  will  require  more  concentration  in  all  areas  of  activity,  whereas 
the  renewal  assessment  should  concentrate  on  problems  previously  noted. 

When  preparing  for  an  assessment,  the  TE  should: 

1.  Notify  the  applicant  at  least  one  month  in  advance. 

2.  Discuss  with  the  applicant  by  telephone:  the  laboratory  location, 
directions,  dates,  time  of  arrival,  security  arrangements,  names  of 
laboratory  staff  who  will  be  involved  in  the  assessment,  suitable  lodging 
arrangements,  proposed  agenda,  requested  demonstrations  of  specific 
tests . 

3.  Request  that  the  applicant  send  appropriate  information  for  the  TE  to 
review  before  the  visit  (e.g.,  QA  manual  and  procedures).  Explain  to  the 
applicant  that  any  documents  sent  in  advance  will  be  held  in  confidence 
and  will  be  returned  upon  completion  of  the  assessment. 

4.  Review  all  relevant  information  (e.g.,  criteria,  test  methods,  QA 
manual,  application  information,  previous  assessment  reports, 
deficiency  correction  letters,  proficiency  test  results).  Prepare 
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questions  based  on  previous  assessment  reports  and  correspondence  to 
address  weaknesses  previously  observed.  Review  any  proficiency  test 
results  and  look  for  trends  that  might  indicate  weakness  of  any  apparatus 
or  of  procedures.  This  information  may  be  used  to  decide  what  tests  need 
to  be  demonstrated. 

5.  Develop  an  agenda.  If  time  allows,  send  the  proposed  agenda  to  the 
applicant . 

V.  ON-SITE  ASSESSMENT 

The  on-site  assessment  is  an  extremely  important  element  of  the  evaluation 
process.  An  assessment  is  conducted  before  initial  accreditation  and 
periodically  thereafter.  Assessments  may  last  from  one  to  several  days  and 
may  involve  more  than  one  assessor  depending  on  the  scope  of  accreditation 
desired  by  the  applicant. 

Checklists 


The  assessment  is  conducted  with  checklists,  which  are  used  to: 
ensure  uniformity  of  assessments  among  assessors; 
guide  each  assessment; 

provide  a place  for  recording  notes  during  an  assessment; 

help  assessors  prepare  exit  briefings  and  assessment  reports; 

provide  physical  evidence  that  an  assessment  was  performed  and  that  all 
applicable  aspects  of  the  laboratory  were  examined;  and 

provide  information  for  future  assessments,  thus  facilitating  continuity. 

The  structure  and  content  of  assessment  checklists  differ  with  technical  area. 
Specific  instructions  are  provided  with  each  set  of  checklists. 

Some  general  tips  in  completing  the  checklists: 

Be  sure  that  each  item  on  the  checklists  has  been  completed  or  that 
there  is  a definite  indication  that  no  answer  was  feasible  or 
applicable . 

Keep  as  many  notes  as  possible  on  the  checklists;  if  there  is 
insufficient  space,  use  additional  sheets  of  paper; 

Although  many  of  the  items  on  the  checklists  are  arranged  in  a 
suggested  sequence,  the  sequence  may  be  altered  as  necessary. 


4 


Typical  Steps  of  an  Assessment 

After  the  entry  briefing,  an  assessment  may  be  performed  in  any  order.  It  is 
generally  advisable  to  follow  the  checklist  sequence,  but  not  always  required. 
Some  assessors  plan  a sequence  of  activities  with  which  they  are  more 
comfortable.  The  purpose  of  the  assessment  is  to  verify  that  the  laboratory 
complies  with  all  applicable  NVLAP  accreditation  requirements. 

During  the  assessment,  the  laboratory  contact  person  should  be  kept  informed 
of  any  deficiencies  that  are  uncovered  so  that  corrective  actions  can  be 
effected  as  soon  as  possible,  perhaps  even  prior  to  the  exit  interview.  This 
policy  avoids  surprises  at  the  exit  interview,  establishes  good  rapport,  and 
makes  for  a smoother  assessment. 


A typical  sequence  of  an  assessment  follows: 

1.  Conduct  an  entry  briefing.  Upon  arrival  at  the  laboratory  to  be  assessed: 

meet  with  the  laboratory's  authorized  representative  and  all  others  who 
will  assist  with  the  assessment; 

explain  the  purpose  of  the  assessment  and  announce  the  agenda; 

request  an  escort  who  is  knowledgeable  about  the  laboratory's 
management , quality  assurance,  and  procedural  systems  to  ensure 
cooperation  of  laboratory  personnel; 

confirm  that  any  laboratory  documents  previously  reviewed  are  still 
current ; 

request  a place  to  review  laboratory  documentation  and  records,  to 
complete  the  checklists,  and  to  compose  findings; 

discuss  the  schedule  for  the  day,  conforming  as  far  as  possible  to  the 
laboratory's  working  hours,  lunch  hour,  and  coffee  breaks; 

agree  on  a tentative  time  for  holding  the  exit  briefing; 

write  down  all  names  and  positions  of  relevant  persons  for  future 
reference ; 

2.  Walk  through  all  areas  of  the  laboratory  where  testing  activities  are 
carried  out  pertinent  to  the  requested  accreditation  to  become  acquainted 
with  the  layout  and  to  meet  the  cognizant  laboratory  personnel. 

3.  Examine  QA  manuals,  personnel  competency  records,  equipment  maintenance 
and  calibration  records,  sample  handling,  and  recordkeeping  procedures. 

4.  Examine  test  plans,  reports,  data  logs,  and  related  records. 
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5.  Trace  one  or  more  samples  through  the  laboratory  process,  from  receipt  to 
final  test  report.  Three  possible  approaches  are  to: 

trace  a test  through  the  testing  process  forward  from  receipt  of 
sample  to  final  test  report; 

trace  a test  back  from  final  test  report  to  receipt  of  sample;  or 

randomly  select  certain  files,  data  sheets,  reports,  etc. 

The  first  two  approaches  are  generally  more  efficient  and  effective  than  the 
third.  Indeed,  some  questions  on  the  checklists  are  sequenced  for  these  two 
approaches.  The  third  approach  should  be  used  when  a large  number  of  test 
methods  must  be  covered. 

6.  Witness  demonstrations  of  selected  procedures  by  laboratory  staff. 

These  should  usually  be  performed  by  the  individuals  who  normally  do 
them,  not  by  the  supervisor  or  manager. 

7.  Interview  technicians.  If  possible,  conduct  these  interviews  in 
private  with  the  supervisor  not  present. 

8.  Physically  examine  equipment  and  facilities.  Observe  the  equipment 
for  appropriateness,  general  appearance  and  working  condition.  Do 
not  operate,  disassemble  or  otherwise  manipulate  the  equipment.  Ask 
the  regular  operator  to  perform  any  operation,  adjustment  or 
disassembly  required  to  establish  evaluation  conditions. 

9.  Complete  the  assessment  report.  Review  all  checklist  items,  notes 
and  other  information  collected.  Summarize  all  findings  on  the 
report  form  and  add  anything  thought  to  be  significant,  whether 
complimentary  or  negative.  Specify  any  items  that  are  deficient  and 
require  a response  from  the  laboratory. 

Be  sure  to  discuss  any  observed  deficiencies.  Sometimes  further  information 
can  be  provided  and  change  first  impressions.  If  a deficiency  is  found,  ask 
questions  to  discover  the  cause.  After  determining  the  underlying  cause, 
suggest  corrective  action  that  might  help  to  satisfy  requirements.  A 
constructive  suggestion  for  corrective  action  is  likely  to  be  acted  upon 
promptly  by  the  laboratory,  particularly  if  it  is  easy  to  implement  and  holds 
promise  of  being  effective. 

Note  whether  the  laboratory  is  already  aware  of  a deficiency.  If  so, 
determine  what  action  has  been  taken  to  determine  the  cause  and  to  implement 
corrective  action. 

Some  deficiencies  are  more  critical  than  others.  It  is  virtually  impossible, 
however,  to  provide  decision  rules  for  objectively  ranking  deficiencies.  Here 
the  assessor's  expert  technical  judgment  comes  into  play. 
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Some  typical  causes  of  deficiencies  can  be  categorized  as  follows: 

Measures  have  not  been  taken  to  ensure  that  an  activity  is  performed, 
or  that  it  is  performed  properly. 

Lack  of  indoctrination/training  of  the  personnel 

Lack  of  time --too  much  pressure  of  work,  overwork,  or  inadequate 
manpower . 

Lack  of  resources- - incorrect  equipment. 

Lack  of  top  management  support. 

Typical  deficiencies  include: 

Required  items  in  the  criteria  are  not  addressed. 

Contents  of  instructions  are  not  adequate. 

Procedures/test  plans  do  not  have  sufficient  detail. 

Outdated  copies  of  documents  are  still  in  use. 

Procedures  described  in  the  QA  manual  or  instructions  have  been 
ignored. 

Samples/specimens  bear  no  identification  to  permit  traceability. 

Responsibility  for  keeping  documents  updated  has  not  been  assigned. 

No  planned  maintenance  records  of  equipment  are  available. 

Equipment  is  out-of- tolerance  and/or  has  not  been  calibrated  or 
verified  at  requisite  intervals. 

Cleanliness  is  inadequate. 

No  provision  is  made  for  protected  storage  in  the  laboratory. 

Substitute  equipment  is  used  without  acceptable  evidence  that  it 
performs  as  specified  by  the  pertinent  standard  involved. 

Supervision  is  inadequate. 

On-the-job  practice  is  careless. 

Personnel  are  not  adequately  trained  to  perform  their  tasks. 

Objective  evidence  that  attests  to  the  training/certification  of 
competence  of  the  personnel  is  not  available  in  laboratory  files. 
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10.  Conduct  an  exit  briefing. 

The  laboratory  will  decide  who  attends  the  exit  briefing.  However,  the 
assessor  should  request  that,  as  a minimum,  the  laboratory's  QA  manager  or 
equivalent  and  the  laboratory  supervisor  should  attend.  For  future  reference, 
the  assessor  should  identify  the  attendees  at  the  exit  briefing  on  the  first 
page  of  the  assessment  report. 

For  each  deficiency,  describe  the  finding,  the  justification  or  reason  for 
stating  the  finding  and,  if  appropriate,  the  proposed  corrective  action.  Make 
sure  that  the  laboratory  understands  each  deficiency  and  what  is  required  to 
achieve  compliance. 

Some  identified  deficiencies  may  be  challenged.  The  laboratory Is  technical 
staff  are  also  experts  hence  differences  of  opinion  may  arise.  Try  to  resolve 
disagreements  and  be  satisfied  that  the  NVLAP  criteria  have  been  met.  If  the 
laboratory  is  adamant  about  a contrary  position,  don't  argue:  ask  them  to 
document  their  position  in  a written  response  to  NVLAP.  The  NVLAP 
accreditation  process  provides  a second  level  evaluation,  including  a review 
by  other  technical  experts  and  NVLAP  staff  of  each  assessment  report. 

Advise  the  laboratory  to  write  NVLAP  within  30  days  describing  action  taken  or 
planned  to  be  taken  for  each  specific  deficiency  identified  in  the  assessment 
report.  The  letter  must  be  signed  by  the  authorized  representative  of  the 
laboratory,  who  can  verify  the  statements  made  in  the  letter. 

Laboratories  assessed  for  initial  accreditation  must  correct  or  provide  a 
substantive  plan  for  correction  of  all  deficiencies  prior  to  accreditation. 

If  considerable  time  will  be  required  to  meet  the  criteria,  they  may  request 
that  NVLAP  hold  their  application  inactive  until  they  are  ready  to  proceed. 

If  any  deficiencies  are  noted  at  laboratories  which  are  currently  accredited, 
such  deficiencies  must  be  corrected  within  30  days  or  they  may  face  suspension 
or  revocation  of  accreditation.  When  out-of-calibration  apparatus  is  cited, 
emphasize  that  the  apparatus  should  not  be  used  until  corrective  action  has 
been  completed. 

During  the  exit  briefing,  watch  for  warning  signs  of  difficulties  in  obtaining 
compliance.  Normally,  the  management  should  be  advised  of  all  problems 
encountered  at  the  laboratory.  However,  in  unusual  circumstances,  a separate 
follow-up  note  to  NVLAP  should  be  considered  if  major  problems  at  the 
laboratory  could  not  be  openly  or  comfortably  discussed  with  the  laboratory's 
management . 

Make  any  necessary  revisions  or  additions  to  the  assessment  report  in  light  of 
the  exit  briefing.  At  least  one  laboratory  representative,  preferably  the 
authorized  representative,  must  sign  the  assessment  report  indicating  that  it 
has  been  discussed  during  the  exit  briefing.  Leave  a copy  of  the  report  with 
the  person  who  signs  it  or  allow  the  laboratory  to  photocopy  the  report.  Do 
not  leave  a copy  of  the  checklists. 
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After  leaving  the  laboratory,  mail  the  original  copy  of  the  checklists  and  the 
signed  original  of  the  report  to  NVLAP.  Retain  a copy  of  all  sheets  in  case 
the  originals  are  lost  in  the  mail,  and  to  refer  to  during  the  technical 
evaluation.  After  an  accreditation  decision  is  made  by  NIST,  return  all 
documents  pertaining  to  the  assessment  to  NVLAP.  Mail  all  documents  to: 


NVLAP 

National  Institute  of  Standards  and  Technology 
Bldg.  411,  Room  A124 
Gaithersburg,  MD  20899 


Monitoring  Visits 

In  addition  to  regularly  scheduled  on-site  visits,  an  assessor  may  be  asked  to 
conduct  monitoring  visits.  These  may  be  scheduled  for  cause  or  on  a random 
selection  basis  during  the  accreditation  period.  These  visits  serve  to  verify 
reported  changes  in  the  laboratory  facilities  and  operations  or  to  explore 
possible  reasons  for  poor  performance  in  proficiency  testing.  The  scope  of 
the  monitoring  visit  may  range  from  checking  a few  designated  items  from  the 
checklists  to  a complete  review.  An  assessment  report  is  written  and  left 
with  the  laboratory  as  is  done  during  a regular  assessment,  but  only  pertinent 
checklist  items  need  be  covered  or  completed. 

When  an  assignment  is  made  for  a monitoring  visit,  NVLAP  will  specify  what 
should  be  examined,  but  the  assessor  has  considerable  latitude  to  make 
judgments  about  other  items  that  should  be  reviewed.  Monitoring  visits 
generally  last  no  longer  than  one  day. 


General  Assessment  Conduct  and  Technique 

A successful  assessment  will  require  more  than  technical  expertise. 
Courtesies  to  be  observed,  attitudes  to  be  taken,  ethical  issues,  assessment 
concepts,  and  questioning  techniques  all  come  into  play. 

While  conducting  an  assessment,  an  assessor  should: 

Conform  to  the  normal  operating  schedule  of  the  laboratory  insofar  as 
possible . 

Be  determined,  decisive  and  direct. 


Keep  the  assessment  moving  and  be  aware  of  the  overall  progress  to 
avoid  wasting  time.  Having  obtained  enough  evidence  to  form  the  basis 
for  a sound  judgement,  there  is  no  point  in  going  over  the  same  ground. 

Be  honest  and  fair.  Personal  dislike/prejudices  must  not  interfere 
with  the  assessment. 
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Be  independent.  The  assessor  decides  what  will  be  examined. 

Recognize  the  difference  between  a "clean-up"  job  and  a "cover-up"  job. 

Discuss  and  clarify  problems  right  away  to  deter  later  arguments  after 
memories  have  grown  hazy,  then  collect  new  information.  If  the 
discussion  reaches  an  impasse,  do  not  argue  but  proceed  to  the  next  item. 

Be  prepared  to  return  to  an  area,  if  necessary,  to  obtain  new  information 
and  to  reassess  the  operation  in  the  light  of  new  information. 

Keep  a sense  of  proportion.  Keep  the  magnitude  and  significance  of  a 
deficiency  in  perspective.  Don't  pursue  trivial  matters  which  waste 
time  and  effort  and  risk  alienating  the  laboratory. 

Be  constructive  at  all  times.  If  a deficiency  is  found,  an  assessor 
may  suggest  possible  corrective  action. 

Admit  mistakes  when  wrong;  this  elicits  cooperation  and  respect  from 
the  laboratory. 

Be  helpful.  Suggest  procedures  that  may  be  beneficial  to  the 
laboratory.  Such  helpful  suggestions  need  not  be  NVLAP- oriented.  Be 
careful,  however,  not  to  give  away  another  laboratory's  "trade  secret" 
in  an  attempt  to  be  helpful.  Open  literature  sources  are  preferred. 

At  the  end  of  each  day,  give  the  laboratory  representatives  a brief 
summary  of  activities,  as  a courtesy  measure.  It  limits  the  need  to 
make  embarrassing  revelations  at  the  exit  briefing  and  in  the 
assessment  report  and  forewarns  the  laboratory  about  any  deficiencies 
encountered.  A deficiency  observed  early  in  the  assessment  can  often 
be  corrected  before  the  exit  briefing. 

Try  to  answer  all  questions  posed;  if  you  lack  information,  offer  to 
provide  an  answer  at  a later  date,  but  be  sure  to  follow  up  or  to  refer 
questions  to  the  NVLAP  staff. 

Thank  the  laboratory  for  its  assistance  and  hospitality.  Even  if  there 
were  some  contentious  issues  and  differences  indicate  that  you 
appreciated  the  laboratory's  cooperation. 

Treat  privileged  information  as  such. 

Avoid  all  situations  which  might  be  construed  as  undue  or  improper 
influences  on  your  assessment  findings. 

Do  not  request  favors  from  laboratories  which  can  be  construed  as 
improper  or  an  imposition. 
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Plan  to  pay  for  all  meals.  In  some  cases,  the  only  eating  facility  is 
a laboratory-operated  cafeteria.  Lunch  supplied  by  a laboratory  in 
these  cases  is  acceptable.  However,  always  offer  to  pay. 

Avoid  making  derogatory  remarks  about  specific  manufacturers  or 
suppliers  of  equipment  or  about  their  products. 

Do  not  promote  any  specific  vendors  products. 

Avoid  making  derogatory  remarks  about  individuals  either  within  or 
outside  the  laboratory. 

Avoid  becoming  involved  in  intra- laboratory  personnel  problems. 


Information  Collection 


Assessment  is  an  information  collection  process  that  includes  verification  of 
application  information;  questioning  staff;  examining  facilities,  equipment, 
and  records;  and  reviewing  applicable  proficiency  testing  results.  An 
assessor  should  collect  information  by  observing  and  listening  (allowing 
laboratory  staff  to  do  most  of  the  talking).  While  conducting  the  assessment, 
please  note  the  following: 

key  persons  interviewed; 

applicable  document  designations,  revision  dates,  where  found,  and 
descriptions ; 

equipment  numbers  and  identification  to  cross  reference  with  calibration 
and  maintenance  records ; 

sample/specimen  identification  system; 

identification  of  document/information/equipment  recipients  in 
laboratory ; 

flow  charts  showing  how  the  laboratory  functions  in  terms  of 
input(sample  receipt),  prerequisites  (people,  equipment,  etc), 
processing  (testing  procedures) , and  output  (test  report) , 
latest  revision  dates  in  the  QA  manual,  procedures,  and  instructions. 

Do  the  laboratory  staff  members  know  their  jobs? 

Do  the  lab  managers/supervisors  want  to  answer  all  questions?  Some  may 
want  to  address  them  all,  either  fearing  that  subordinates  will  commit 
an  error  that  reflects  adversely  on  the  laboratory,  or  because  they 
feel  it  is  their  right  and  responsibility  to  be  the  knowledgeable 
spokesman.  Make  it  clear  that  the  "worker"  should  answer  the 
questions,  explaining  that  the  purpose  of  the  assessment  is  to 
determine  what  is  actually  being  done  and  to  verify  that  the  procedures 
are  implemented  as  indicated  in  the  quality  manual. 
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Does  the  staff  try  to  bluff  its  way  out  of  a tight  corner? 

Does  the  staff  know  the  QA  manual  and  associated  procedures  and  where 
these  documents  are? 

What  is  the  condition  of  facilities  and  equipment?  Are  the  requisite 
calibration  tags  or  stickers  on  the  equipment? 

Does  it  appear  that  the  QA  and  procedural  documents  are  used? 


Questioning  Techniques 

Effective  questioning  is  vital  to  an  assessment's  success.  It  is  of  great 
value  to  keep  the  systems  concept  in  mind,  to  separate  the  laboratory's  work 
into  input,  prerequisites,  process,  and  output,  and  to  ask  questions  on  each 
of  these  individually  and  in  turn.  Verbalizing  an  understanding  of  how  the 
system  works  will  allow  the  laboratory  to  correct  any  misconceptions. 

Ask  questions  that  require  a substantive  response  (e.g.,  "Who,  What,  How, 

When,  Where,  and  Why"  questions  force  a substantive  response),  rather  than  a 
"Yes/No"  answer.  Questions  that  require  a "Yes/No"  answer  are  useful  to 
clarify  particular  points.  Statements  or  questions  such  as:  Show  me  what  you 

do. /Where  do  you  put  that?/How  do  you  do  that?/  are  recommended.  If  there  is 
a variation  from  written  procedures,  then  ask  "Why  do  you  do  it  that  way?" 

Hypothetical  questions  are  good  when  little  objective  evidence  is  available. 
"What  if..."  or  "Let  us  suppose..."  or  "I  don't  understand..."  is  usually 
effective  in  such  situations.  Avoid  questions  that  suggest  the  expected 
answers . 

Silence  can  be  extremely  powerful.  When  faced  with  a silent  person,  some 
people  may  feel  a response  is  expected,  or  because  they  are  not  quite  sure 
what  the  response  should  be,  they  may  say  more  than  they  otherwise  would. 

This  often  produces  useful  information. 

Be  aware  of  what  the  laboratory  staff  does  not  say.  If  answers  are 
superficial  or  evasive,  be  prepared  to  continue  the  line  of  questioning. 

Be  alert  to  differences  in  information  presented  from  different  sources  (i.e., 
conflicting  answers  from  staff  members  or  differences  between  documented 
procedures  and  what  is  actually  observed  or  said) . 

Attempt  to  put  the  staff  at  ease  at  the  beginning  of  a questioning  session 
since  they  will  probably  be  nervous  and,  if  so,  may  become  withdrawn  or 
defensive,  or  worse- -by  responding  aggressively.  Do  not  be  aggressive  in 
return,  but  be  thorough  (and  occasionally  persistent)  enough  to  establish  the 
factual  situation.  Keep  the  following  points  in  mind: 
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Direct  questions  to  the  person  who  performs  the  task  being  assessed,  not 
to  that  person's  superior. 

Never  talk  down  to  anyone . 

Talk  the  " lab's  language". 

Give  credit  where  credit  is  due.  A compliment,  sincerely  given,  goes 
a long  way  towards  eliciting  cooperation. 

Be  interested  in  the  laboratory's  work  and  responses. 

Don't  appear  to  be  distrustful  of  people  or  to  regard  their  responses 
with  criticism. 

Be  calm  and  courteous  and  thank  people  for  their  time. 


VI.  TECHNICAL  EVALUATION 

A technical  evaluation  (second  level  review)  of  all  applicants  is  performed 
prior  to  granting  accreditation.  Normally,  one  or  more  TEs  who  were  not 
involved  in  the  assessment,  along  with  NVLAP  staff,  review  the  evaluation 
folder  for  each  laboratory.  The  evaluators  provide  technical  judgments  on 
whether  an  applicant  complies  with  the  conditions  and  criteria  for 
accreditation  and  make  recommendations  on  appropriate  accreditation  actions. 

In  some  cases,  a special  form  is  used  to  record  TE  evaluation  recommendations. 
A TE  recommendation  is  based  on: 

information  provided  on  the  application; 

assessment  reports; 

actions  taken  by  the  laboratory  to  correct  deficiencies  observed  during 
the  on-site  visit; 

results  of  proficiency  testing  from  current  and  previous  assessment 
periods;  and 

information  from  any  monitoring  visits. 

If  a laboratory  is  not  considered  to  be  in  compliance  with  all  requirements, 
it  is  notified  with  a request  for  additional  information  to  demonstrate 
compliance.  An  additional  assessment  may  occasionally  be  scheduled. 

The  NVLAP  staff  prepares  the  paperwork,  including  the  transmittal  of 
correspondence  to  any  laboratory  for  which  the  evaluators  need  additional 
information  concerning  deficiencies  that  must  be  resolved  before  accreditation 
can  be  recommended. 
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When  a laboratory  is  notified  about  specific  deficiencies,  it  must  respond 
within  30  days  to  provide  documentation  or  certification  by  an  authorized 
member  of  management  that  the  specified  deficiencies  have  been  corrected. 
Clarification  of  some  issues  may  be  requested  by  telephone.  All  deficiencies 
must  be  corrected  before  accreditation  is  granted. 

Whenever  an  applicant  laboratory  satisfies  all  requirements,  a certificate  and 
scope  of  accreditation  is  sent  to  the  laboratory. 


SUMMARY 

NVLAP  is  a system  which  provides  fair,  unbiased  evaluation  of  the  competence 
of  testing  laboratories  to  provide  various  but  specific  testing  services. 
Rigorous  criteria,  are  rigorously  applied.  The  resulting  accreditation 
benefits  both  the  accredited  laboratory  and  the  potential  user  of  laboratory 
services  who  may  use  accreditation  as  a selection  factor  that  will  inspire 
confidence . 

The  NVLAP  Technical  Experts  play  an  extremely  important  role  in  the 
accreditation  process,  both  as  on-site  assessors  and  as  technical  evaluators. 
To  carry  out  their  functions  these  individuals  must  possess  extensive 
technical  knowledge  and  excellent  communication  skills. 

The  principles  and  techniques  described  in  this  manual,  if  understood  and 
followed,  will  provide  a solid  basis  for  performing  the  functions  of  a NVLAP 
technical  expert.  An  individual  who  represents  NVLAP,  is  expected  to 
demonstrate  the  highest  standards  of  competence,  character  and  ethical 
conduct . 
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